Permutation p-values for specific-category kappa measure of agreement
نویسندگان
چکیده
منابع مشابه
Fuzzy kappa for the agreement measure of fuzzy classifications
Fuzzy kappa for the agreement measure of fuzzy classifications Weibei Dou , Yuan Ren, Qian Wu, Su Ruan, Yanping Chen, Daniel Bloyet, JeanMarc Constans Department of Electronic Engineering, Tsinghua University, 100084 Beijing, China GREYC-CNRS UMR 6072, 6 Boulevard Maréchal Juin, 14050 Caen, France cCReSTIC, 9 Rue de Qubec,10026 Troyes, France Imaging Diagnostic Center, Nanfang Hospital Guangzho...
متن کاملA-Kappa: A measure of Agreement among Multiple Raters
Abstract: Medical data and biomedical studies are often imbalanced with a majority of observations coming from healthy or normal subjects. In the presence of such imbalances, agreement among multiple raters based on Fleiss’ Kappa (FK) produces counterintuitive results. Simulations suggest that the degree of FK’s misrepresentation of the observed agreement may be directly related to the degree o...
متن کاملKappa coefficient: a popular measure of rater agreement
In mental health and psychosocial studies it is often necessary to report on the between-rater agreement of measures used in the study. This paper discusses the concept of agreement, highlighting its fundamental difference from correlation. Several examples demonstrate how to compute the kappa coefficient - a popular statistic for measuring agreement - both by hand and by using statistical soft...
متن کاملPermutation P-values should never be zero: calculating exact P-values when permutations are randomly drawn.
Permutation tests are amongst the most commonly used statistical tools in modern genomic research, a process by which p-values are attached to a test statistic by randomly permuting the sample or gene labels. Yet permutation p-values published in the genomic literature are often computed incorrectly, understated by about 1/m, where m is the number of permutations. The same is often true in the ...
متن کاملEstimation of false discovery rate using sequential permutation p-values.
We consider the problem of testing each of m null hypotheses with a sequential permutation procedure in which the number of draws from the permutation distribution of each test statistic is a random variable. Each sequential permutation p-value has a null distribution that is nonuniform on a discrete support. We show how to use a collection of such p-values to estimate the number of true null h...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Journal of the Korean Data and Information Science Society
سال: 2016
ISSN: 1598-9402
DOI: 10.7465/jkdi.2016.27.4.899